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ABSTRACT 

This study vras concerned with various schemes for 
reducing the numuer of variables in a mult i vat iate analysis. Two sets 
ot illustrative data were used; the numbers of criterion groups were 
3 and 5. The proportion of correct cla.ssif ica*^ ions was employed as an 
index of d iscrimina tory power of each subset of variables selected. 

Of the four procedures using indices that order the variables with 
respect to contribution to discrimination, the (forward) stepwise 
procedure yielded the best results. Ot the two schemes involving 
dimensional analysis, that which uses correlations of scores on 
variables with high maximum likjlihood factor loadings again.t 
discr iminant scores appeared more attractive. (Author) 
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Abstract 

This study was concerned with various schemes for reducing the number of 
variables in a multivariate analysis. 2 sets of Illustrative data were usee 
the nimibers of criterion groups were 3 and 5. The proportion of correct classifi- 
cations was employed as an index of ciscrimiratory power of each subset variables 
seie('ted. Of the 4 procedures using indices that order the variables with respect 
to contribution to discrimination, the (.forward) stepwise procedure yielded the 
best results. Of the 2 schemes involving dimensional analysis , that which uses 
correlations of scores on variables V7i th high maximum likelihood factor loadings 
against discriminant scores appeared more attractive. 
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ON THE VARIABLE SELECTION PROBLEM 
IN MULTIPLE GROUP DISCRIMINANT ANALYSIS 

Carl J Huberty^ 

University of Georgia 

In many situations invol\ ing multiple group discriminant analysis the 
investigator is presented vith more variables than he would like and there 
arises the question of whetVier they are all necessary and, if not, which of 
them can be discarded. Having obtained weights which determine the discrimi^ 
nant scores, the Investigator may ask if the data might not have been ade- 
quately explained by using only a subset of the original input variables, Whe i 
an experimenter is confronted with such a problem he wants to include as many 
variables as possible so that reliable values may be determined, and yet as 
few as possible so as to keep the costs involved in obtaining information at 
a minimum. 

For the case of a discriminatory analysis involving more than lwo criterion 
populations, no knovm optimi^ing procedure has yet been developed to reduce the 
number of discriminator variables. (Optimum in the sense that the variables 
selected would lead to a maximum amount of separation among the groups for that 
number of variables.) There is a dearth of literature concerning the reduction 
in number of variables in discriminatory analysis. To date few studies have 
used any selection procedures in actual research and fewer still have subjected 
any methods to empirical comparisons. Until recently, of course — i.e,, before 
tlectronic computers became readily avallabl >--the computation' involved in 
discriminatory analysis and the deletion of variables proved to be a task of 
such herculean proportions that c»n investigator could hardly be blamed tor not 
planning his study to include such an analysis. 
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Quenoville (1949a) developed a conjputational scheme tor deleting variables 
when the under Jving model was that of Fisher's, in which, for more chan two 
criterion groups, colllnearlty of population means is assumed^ and thus only 
one discriminant function is obtained. His scheme involves a fjrsc approxi- 
mation to the eigenvector associated with the largest corresponding eigenvalue, 
and a process of successive iterations which leads to convergence of the dis- 
criminant function coefficients. Thus, he arrived at a procedure whereby the 
initial analysis is used to avoid the complete recalculation of a new dis- 
criminant function when potential discriminators are discarded. In a second 
/ 

article (1949b) he eliminated the Difficulty of Caking special action to prevent 
the coefficients from converging to zero. 

Rao (1952, p 253; 1965, p. 482) presents a statistic which may be used 
as a criterion for discarding p-q potential discriminators. The statistic 

^ ^2 - P - ^ 

p-q (N^ f N 2 ) (Nj^ + - 2) + 

where is Mahaianobls' estimate of Che distance between populations i and 2 
based on the original set of p measures, has an approximate F distribution with 
(p-q) and (N^ + ** p - i) degrees of freedom. This statistic may be employed 

to test the hypothesis that (p *q) measures on the variables do not provide additional 
discrimination. This test Is applicable for only two p-varlable populations at 
at a tine (from which random samples of size and ^2 selected). 

Collier (1963) shov;s that since there Is a direct relationship between 
regression analysis and dlscritrilnaut analysis for the two- group case, methods tor 
deleting variables In regression analysis could be carried over to drsctiminart 
analysis. He displayed the equivalence between the regression test (Involving the 
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iHuxiiple coirej.acion v,oefficienc; and che above scatiscic ji Rao s rot cesLing 
the wuntiibuLLon ol acldiLional variables, 

Co.hran and Bliss (1948) ernpiO)' a covariance technique (ror use wich cw^ 
liLierion populations) to deLecmine if a particular discrimin-jcor . or subset 
Dt discriminators, contribute anything to the adjiistea discriminant In thxs 
analysis the questionable discriminators are treated as the cc.ariaces An 
estimated standard error (based on an error mean square ot an ANOVA; oi d -«,oet t ic lent 
IS found which is used in forn,ing. a c ratio, Thus, if omission of some disorii'ii- 
natets seems warranted, these t ratios are relevant in deciding whi^h va.'idbie 
CO eliminate first 

Again considering the two-group case and a single discr imnanti Kenddii 
(i957, p 163) gives the standard error (for large sampies; or the ^oet i i>.icnts 
in the linear disciiminanc . Thus ; using a simple t kCSt it be determined ii 

the variable corresponding to a sample coefficient can be discarded without d 
seriou§ loss of discriminating power Grimsley and Summers (l963) used such a 
ce.:hnique in a study to delete three of lOur variables. 

Some ocher attempts have been made to us methods whi^h the number ot 
potential discriminators is reduced before computing a discriminant, the 
grounds that their Inciusicn is unlikely to produce a material increase in power 
to discriminate between two groups Horst and Smith (l950) were able vo ais, 

? of 18 original variables on whxch physical measures were used di t i e rent late 
between men of Japanese and Caucasian stock Wallace and Ttaiers (i9o&> used 
only 5 of more than 20 available variables in distinguishing successful from 
unsutcessfui salesmen In both situations the criterion used to retain a 
variable in the analysis was che significance (at the Oi level) rendered by 
a t test of difference between the two population means of the callable in question 
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Cochran (1964) 'es not favor this latter test since the t-vaiue is gceatiy 

influenced by the sizes of the samples that are selected for setting up the 

discriminant. On the contrary he states that . .if it is the fact that in 

practice most correlations arc positive and modesr in size, the analysis suggests 

2 

that reliance or. the value of fd^ in deciding whether to throw away a group ot 

poor discriminators is unlikely to produce a serious mistake [p l86j.’' The value 

is defined to be the squared normalized difference of the means ot variable 1 

in the two criterion populations. Twelve examples from the literature indicated 

2 

that it will usually be safe to reject a group of variables if the value of id^ 

IS small. Because of the relatively small sample sizes used in the examples cited 
the conclusion ii.ay be somewhat questionable - 

Procedures such as these have limited practicality in that they do not aid 
in determining which discriminators to delete or which to retain in a k-group 
(k 2) situation. Dunteman (1966) reduced the number of SVIE scales from 29 ti li 
in a study where these scales were used to discriminate female students among five 
occupational groups He employed two d^^fferent approaches to delete variables. 

The first approach was to use in the final analysis only those scales which had 
produced univariable F-ratios that were significant beyond the 01 ievei The 
second criterion used was that of selecting the variables which had relatively 
high weights on one or more of the discriminants obtained Using the nur.iber oi 
correct classifications as a criterion of effectiveness the variables selected, 
his results shov/ed that both of the variable reduction procedures resulted in 
about the same amount of efficiency as when including all of the original Z9 
variables DeMann (1963) employed univariate chi-square tests to delete x/ out 
of 20 potential discriminators prior to his muitivaiiane predictive analysis 

The classical method of finding clusters of variables which are formed under 
certain consistent principles of classification Is factor analysis, another 

o 
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muiiivariate proceduie„ The purpose of factor analysis is the siady ot oependeii^c 
parterns in rhe variables. This is accomplished by seeking actiti^ai vdCiabies 
(ta^torsl which may explain the dependence among the obsei-vahie variables Jt 
IS desirable to keep the number ot such artificai variables as small aS possible 
For greater ease in interpretation of multivariate data, factor analysis is 
sometimes used to reduce the number ot variables, "rr>egh, cl n i {V}'z} ) loiiowed 
this scheme in an attempt to determine the nature of masculinity and femininity 
in the preschool years; four criterion groups and 60 variables were involved 
A principal axis analysis of a pooled covariance matrix was performed which 
resulted in rhe extraction of 20 factors (approximately i00% ot the estinated 
ccmmunallty was accounted for). Two criteria dictated further reduction to four 
falters; these were rotated and mean factor scores w^ere then estimated fcr c^ll 
tour factors for each criterion group These four "variables" were then sub- 
jectud to a discriminant analysis, This approach to the variable problem is 
inappropriate since to obtain the factor scores it is necessary to stili use 
the scores on all of the origlrial variables Nonetheless, as we shall see 
i<atet, factor analysis methodology may be employed as an aid in selecting <a 
subset of discriminators- 

Selection Procedures 

The present study was ccncerned with various schemes for reduiing the 
number of variables in a multiple group discriminant analysis design The 
analysis referred to is that of deterininlng the eigenvectors associated ';ith 
the eigenvalues (A-values) obtained fren the solution of the equation 
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E - (pxp) pooled within-groups deviation score sum ot 
ot squares and cross products (SSCPJ matrix, 

H = (pxp) among-groups deviation score SSCP matrix, and 

1 = (pxp) identity matrix 

The discriminant weights, then, are the elements of the so determinea (sometimes 
nocmall^ed) eigenvectors. The variable selection criterion most or ten employed — 
implicitly or exp lici t iy-^has been the relative size of the so-caited '^beta weights’’ 
(see, e g,, Clemens, et al (1970J. However, many other schemes are at our dis- 
posal: for example, multiple univariate, stepwise, factor anaiy cic-discriminatory j 

and correlational procedures* The bases on which some selection criteria ace 
founded are briefly discussed in the next paragraphs, 

1) Beta weigh ts The beta weights aie merely the scaled discriminant 
weights — the multiplication factors being the error standard dev’iations of the 
respective variables* A subset of variables is selected by including chose 
variables having large beta weights on the discriminant func t ion (s; --an a£bitrar> 
lower bound for the absolute value may be set where a natural cui-ott uCwUis 

2/ T- tat ios * it has been recommended (eg., GrizzJe, l9*/0> that to deter- 
mine Which variables ought to be subjected to analysis in t.he design ot cin experi- 
ment, einglc variable analyses should be carried out beforehand The usual 
univariate analysis performed is a simple ANOVA with the accompanying v^mnibus 
F-test; f iwever, if the criterion variable responses are categorical and discrete 
In nature, the chi-square statistic may be more appropriate. To determine a 
subset of variables to be used, then, one merely deletes those variables chat 
do not have a reasonable expectation of yielding information about difierences 
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among groups That is > select only those variables that p.oduce oigniLicant 
univariate statistics at, say, the ,0i level Or, selection may be based ^n the 
relative magnitudes of the statistics themselves. 

3) Stepwise valu es Tne stepwise disc nn? nant proc<ju;c thctt outunea 
in the BMD Manual (Dixon, 1967). At each step that variabi'* is selected i; with 
the largest value of an F-statistic, or ii) which when paitiaxcd on the pcevi;.usi> 
entered variables has the highest multiple correlation wich che group:>^ or iiij whx^h 
gives the greatest decrease in the ratio of within to totc^l geru; i a i i 2 ed variance^ 
Also, a variable is deleted at any step it the value of its asso^iaced F-statisti.^ 
becomes too lew. An ordering of :he variabjes ic thus determined, and variables 

may be retained to the point where the value of Walks' lambda ’’levejs ott,’’ or 
when the increase in the proportion of correct classifications is no linger 
"appreciable - " 

4 ) Component loading s. One factor analytic procedure which may be employed 
in variable selection is that suggocted by ho^-st (1965, p 555j Jt involves 

a principal component at^alysis of the matrix of variable int erco r t e J at ions (in 
the present situation an "error" correlation matrix). From a c i ans 1 orma t ion 
(e g., varimax) of the pattern matrix, a subset of variables is selected such 
that each of the components will be adequately represented in the subset 
Variables are selected which have the highest loadings (in obsciv.ie vnlu^; on 
each of the components. Presumably, no variables are selected wlii^h nave 
high loadings on more than one Lcn^ponent, 

5) Factor analvtic-dlscrlml nato ry correlatio ns Bargmann (l96/> recommends 
a n.iximum li!<elihood factor analysis (IfLFA) , with (oblique) rotarion to simple 
structure to define, clusters of varlt:blfes [l.e., factors] which have some 
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underiy ” ig char accer iscic in common. Once a classif icacion of variables int.^ 
;,iusters las been decermined, che problem of '^representacive seiecLion*' or 
variables for the purpose ot discrix"* nation may be solved b> applying the 
general discriminatory techniques to each cluster. These techniques involve, 
for each cluster, finding maximum likelihood estimates of che cor.alations 
between the response variables and the artificial variable (the leaning "dis- 
criminant function") determined by the usual eigenanalysls . Variables are then 
selected that load on each factor and correlate highly with the respe«^tive 
leading discriminant function. It must be noted that consideration is restrict* 
to only the leading discriminant function since this function usually a^-Counts 
tor a major portion of the discriminatory power of the set of predictors 
(Bargmann (1970, p. 55) discusses further reasons for considering only the 
first function in the interpretation of a discriminant analysis j 

6) Varia b le-DF correlations . Some investigators merely order the 
response variables with respect to contribution to the overall discrimination 
between the criterion groups by examining estimates ot the cor rei«it uns of 
the response variable versus the (leading) discr intinanc function. Selection 
IS then based on these so-called "structure" values without employing any 
factor analytic techniques- Two approaches have been used to compute these 
estimates. I^en the concept of "total population" is meaningtui, then the 
p t-values arc determined by the relationship 



UJ 




wh ere 



V = (Ixp) vector of weights for ti.e first discriminant 
function, 



D - (pxp) diagonal matrix of "total" standard deviations 
® i 

of the p vact ab 1 s , 
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R - (pxp) "total’' coti^elation matrix of ihe p predictor 
variables, and 

= (pxp) diagonal matrix of the reciprocal of the 
standard deviation of the scores on the first 
discriminant function. 

Cori'eiatirns computed this way are precisely the r-values that would result it 
the Pearson coefficients were calculated between the sample variable scores and 
the sample discriminant scores [Gulliksen (1950, p, 339)]. If the concept ’'k 
populations equally dispersed" makes sense, then the maximum likelihood estimaLe 
I the true correlation vector is given by [Bargmann (19/0, p. 53) or Porebski 
i966 , p . 266) ] 

r* - (V E vV (v E) D ,, 



where v is defined as above, 



E - (pxp) pooled wl thi n-groups deviation score SSCP 
matrix, and 



D 



l//e 



ii 



(pxp) diag'5n:i matrix of the recipiocals ol the 
positive square roots of the diagonal elements of E 



As with beta weights, the number of variables in the subset seleccea may be 
Jeteimined by a drop in the absolute value of the correlation coef ! i . lent s . 

The major purpose in using the» selection schemes discussed aboce is that 
of determining Indicators of variable potency in terns of accuracy of classifi^ 
cation. Discriminant analysis employed in this sense is strictly descriptive, 
or exploratory in that it may provide leads tor ‘'ibsequcnt investigation 
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To illustrate the application of the above mentioned selection procedures 
two sets of data were used. One set involved measurer Oii 13 variables for three 
samples of college fresmen in a midwestern university for the purpose of assigning 
them to beginning French courses. Table 1 gives the means and standard deviations 
on each variable for each of the three groups. A total of 153 (35 + 31 + 37) 

““Insert Table 1 About Here- 

subjects was involved. Group 1 consisted of those freshmen ’^correccly" assigned 
to the beginning French course. Groups 2 and 3 correspond to students in more 
advanced French coursej. The appropriateness of the level of tne course in French 
for assignment of each of the students was determined by teacher judgment after 
the 1965 fall session ended. More detailed information regarding initial and 
final group assignment procedures is given by Bisbey (1969). 

Seventeen measures on eacVi subject in five educational progress groups cf 
high school students comprise the second set of data. Five small samples 
randomly selected from a nationwide stratified sample of nearly 26,000 eleventh 
gfade students made up the five criterion groups. The sampling produced a total 
of 600 (177 + 26 + 75 + 52 + 270) subjects. The five selected samples were also 
stratified with resp'Jct to sex and educational progress. Measures on the 17 
predictor*; v;ere obtained in 1960, while group membership Wcis determined in 1962. 
Group 1 was comprised of those students who, for two years following high school, 
had not attended any college. Those students v;ho enrolled in a business college 
made up Group 2. Group 3 consisted of vocational college students. The other 
two groups were college enrollees; Group ^ corresponds to students in junlr^r 
college and Group 5 to senior college Students. Means and standard deviations 




12 



- 11 - 



ate presented in Table 2. Variables 1-7 are in the rognitive domain and aci 
measured by various tests of substantial reliability* Measures on Varia!'ies 
6, and 7 are composites of five, two and three tests, respectively* Variables 8-12 

-Insert Table 2 About Here- 

are interest variables, and Vatiables 13-15 are temperament variables. Variable 16, 
"Curriculum,^' is a scholasticism variable, measures on which are ordinal with 
i-agri culture , * , ►, 5=college preparatory. A composite socioeconomic status 

measure was used for Variable 17. 

Res u lts 

ihe results of this study are presented separately for the two sets of data 
Because the analytic techniques used are exploratory ir nature, and for reasons 
dr'scussed by Niannally (1967, p. 388) and Porebski (1966, pp . 2 78-*229), the multi- 
variate tests of mean differences and cf homogeneity of covariance matrices were 
not deemed appropriate here. 

Thr ee-gr oup case. Values of interest for four of the si^'evrim * ' ^ are 

presented in Table 3. Table ^ gives the orderings of the variables according to 

of the criteria employed. Only one "Varj.able vs DF Correlation^' ordering 
is given since the total group , . equation [!)} and v;l tiiin-groopp (sc? equation 
[2]) correlations produced identical order ings. Since nc va^ es are directly 
associated with each variable in the stepwise procedure, the resulting ordering 
is only given in Table 

-Jnfort Tables 3 and About ^ere~ 

The correlation matrix that was analy^ed by the naximan likelihood and 
principal axis pre^fdures w. , the (13 x 13) "error" correlation matrix, R. 
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The elements of this matrix — ’’intrinsic’' correlations — are given hy 



■ij 



JLi 

jj 



where e^^ is the (i-j)th element of the matrix E. By using R, spurious correla- 
tions or "correlations in widespread classes" are ignored [Bargmann (1968, p. 571) J. 
To justify the analysis of R, Bartlett^s sphericity test using a chi-square statis- 
tic (Morrison (1967, p. 113) J, which is a measure of the degree to which R differs 
from 1, was performed. The value of the statistic was 742.349, which for df=?8, 
implies a orobability, under a true null hypothesis, of less than .001 Ll.at this 
value could be attributed to chanca^ The factor analysis program'^ employed in 
this study to obtain, by an iterative procedure, a maximum-likelihood estimate 
of the factor matrix starts with an improved centroid solution. The built-in 
test (a chi-square statistic) for the number of significant factors indicated 
that five factors were adequate. The five resulting factors were then rotated 
obliquely via MAXPLANE [Eber (1966)] in an attempt to arrive at a simple structure 
solution. The rotated factor loadings are given on the left in Table 5. The 

-Insert Table 5 About Here- 
fallowing clusters of variables w^ere determined: 



Factor 


I - 


Variab les 


1. 


2, 


3, 


8 


Factor 


11 - 


Variab les 


5, 


8, 


9, 


10, 11, 12 


Factor 


III- 


Varia'5 les 


6, 


11 


, 12, 13 


Factor 


IV - 


Variables 


2, 


3, 




7, 11 


Factor 


V - 


V^ariables 


2, 


3 


4, 


5, 6, 7, 8 
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Five discriminant analyses were then performed using only the variables 
determining the above factors. Estimates of the correlations between the 
discriminant functions and the variables involved in the functions were cal- 
culated accoiding to equation [2]. The vectors of correlations evaluated for 
each of the five factors are: 



Factor I 



12 3 8 

r* = [ .87, .43, .91, .22] 



Factor II 



5 8 9 10 11 12 

r* = [.48, .03, .15, .15, .67, .84] 



Factor III 



6 11 12 13 

r* = [ , 18, .66, .83, -.22] 



Factor IV 
r* 

Factor V 



2 3 4 7 11 

[.07, .20, .16, .31, .84] 

2 3 4 5 6 7 8 

[.11, .26, .21, .66, .28, .33, .06] 



The orderings of the variables determined by the descending order of the absolure 
values of these correlations are as follo^N's: 



Factor I 
Factor II - 
Factor Ill- 
Factor IV - 
Factor V - 



(3, 1, 2, 8) 

(12, 11, 5, 9, 10) 
(12, 11, 13, 6) 

(11, 7, 3, 4, 2) 

(5, 7, 6, 3, 4, 2, 8) 



To obtain a representative subset of variables, then, ve may choose variables 
1, 3, 5, 11, and 12. The number of variables selected by the other five methods 
was arbitrarily made equal to the number selected hare, namely, five, so that a 
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comparison could be made among the six methods \?hen subsets of the same size 
are selected. 

Two criteria vere considered in determining the number of components to 
extract for the principal axis solution; i) the eigenvalue criterion, and 
ii) the "scree” test [Cattell (1966, p, 206)]. It was decided to extract 
four components. The four eigenvalues found were 4.20, 1.97, 1.58, and 1.06 
(the fifth was 0.76); 68 percent of total variance was accounted for by the 
four components. These components were then rotated to meet the varimax criterion. 

The resulting loadings are given on the right in Table 5. Based on Horst's 
suggestions, and restricting the number to five, che following variables were 
selected: 3, 5, 10, 12, and 13. 

A summary of the five variables selected according to each of the six 
criteria presented in Table 6. 

The classification scheme that was employed in this study is based on the 
posterior probability of group membership [sea Rule 3 in Huberty and Blominers 
preparation (a)]; or Cooley and Lohnes (1962, n. 138; J. The proportions of 
correct classifications when only those variables selected according to the six 
criteria are included are given in Table 6. It was of Interest to test the hypothesis, 

-Insert Table 6 About Here- 

that, for a given sample of subjects, the classification accuracy is Identical when 
using only those variables selected according to the various schemes. Inclusion 
of cill 13 variables--which gave a proportion of .941 — was done to determine if 
the discriminatory pcn^rer is significantly decreased when using only those variables 

selected. A chi-square statistic (Q) suggested by Cochran (1950) was employed. 

2 

For the dichotomous data here the x approximation was Judged satisfactory [Tate 
and Brown (1970)). The overall test of the differences In frequency of correct 

iO 
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classifications resulted in a chi-square value of 26.588 (6 df), "^ith p‘^.0005. 

2 

The test of ’'All 13^' versus the six selected subsets yieid*!d a x <1) “ 12.650* 
with p^.0005. However, when tests were carried out comparing the accuracy of 
the selection cr' eria in a pairwise fashion, not one test statistic was 
significant. And only the test of ^'All 13" versus the principal axis results, 
which yielded a Q-value of 15.385 with a nominal probability of le?^. than .0005, 
was judged significant. 

The number of variables in each subset was determined by the number ^electi’d 
by the : athod involving MLFA. It may be of Interest to investigate the subsets 
yielded by each criteria without this restriction. If the number of variables 
to be selected was not limited to five, the following subsets would probably ha’^e 
been chosen using the respective criteria: beta weights -- 2, 3, , 5, 7, 9, l(i, 

11, 12, 13; F-ratios -- 1, 3, 4, 5, 6, 7, 11, 12, 13; stepwise values -- 2, 3, t, 
5, 6, 7, 9, 11, 12, 13; principal axis loadings — 1, 3, 4, 5, 7, 3, 9, 10, 12, 
13; variable-DF correlations — 1, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13. The correi- 
ponding proportions of correct classifications obtained using these? subsets 
.908, .915, .935, .895, and .928 while the proportion yield(»d by the subset 
determined by the method Involving MLFA Was .882. llie classification accuracy 
yielded by these six subsets along with that yielded by all 13 ,ai:..ables, 
viz., .941, were not found to be significantly different (0 = 10.030, p . 10) . 

Five group case . The same procedures were followed hei e as in the three 
group cast. Some Indices of variable potency for the data involving five 
criterion groups are given in Table ' . Orderings of the 17 variables based on 
four criteria are presented in Table 8, Again the t-^o sets of correlations 

-Insert Tables 7 and 8 Abojt Here- 
of variable scores versus dlscrlminart scores were identically ordered. 
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Bartlett’s test for sphericity yielded a x (136) - 3030.52, with p<.001. 
Thus it was justifiable to analyze the (17 y. 17) error correlation matrix, R, 
by both the maximum likelihood and principal axis methods. The eight resulting 
maximum likelihood factors were rotated so that the 2iAXPLANE criterion was 
met; the loadings are given on the left in Table 9. Separate discriminant 
analyses were performed using only those variables determining the eight 
factors. The ”within-groups'* correlations of those variables involved against 

-Insert Table 9 About Here- 

the resnective leading discriminant functions were then found. The following 
correlation vectors resulted; 

Factor I 



£* = l.94 

Factor II 



1 7 

.87 .14] 



8 



12 10 11 



[.61 -.34 .32 .15] 



Factor 4.11 



6 15 
(.97 .43 



13 

.15] 



Factor IV 



r* = [.99 



3 5 7 

.43 .37 .13] 



Factor V 



16 

r* [.99 



17 

.51] 



Factor VI 



r* - [.59 - 



12 10 11 
,46 .40 .19] 
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Fi^ctor VII 
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6 13 14 

r* = t.99 .16 .38] 



Factor VIII 



6 8 4 5 12 

r* - 1.88 .48 .38 .32 -.25] 



Based on these correlations it was decided that variables 1, 2, 6, 8, 9, 12, and 
16 could be considered a representative subset. The number of variables chosen 
on the basis of each of the other selection criteria was subsequently restricted 
to seven. 

The (varimax) rotated principal axis loadings .^re presented on the right 
in Table 9. Of the six eigenvalues which accounted for 68 percent of the total 
variance, five were greater than unity, On the basio of (the absolute values 
of) the loadings it was decided to chose the following seven variables: 1, 4, 

?, 10, 13, 16, and 17. 

The seven variables selected, based on each of the six criteria, are indicated 
in Table 10, As in the three-group case the comparative efficiency of the selection 

-Insert Table lO About Hare- 

rules was determined by the accuracy Ln classifying the 600 subjects into their 

respective groups. The i>roportion of correct classifications when all 17 variables 

were considered was ,750; propc “^ons ^or the six subsets determined by the 

various criteria are given in Table 10. 

A highly significant {p<.0005) Q-value of 99.642 (6 df) resulted from the 

overall test of the classification accuracy of the six selected subsets and alJ 

2 

1? variables. The test of "All ,7 versus the six subsets yielded a \ (1) ' 
6120.214, w^lth p^.0005. It was fot .J that the accuracy yielded by all 17 
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vaciables was significantly higher than the accuracy yielded by each of the 
six selected subsets, while their was no evidence to conclude that the accuracie 
yielded by the six subsets were different. 

Even when the size of each subset was not restricted to seven and selection 
of subsets was made independently, a significant loss of discriminatory power 
resulted for each subset. The variables selected by five criteria when the 
number was not restricted are as follows: beta weights — 2, 3, 6, 8, 9, 10, 

11, 12, 15, 16; F-ratios — 1, 2, 3, 4, 5, 6, 8, 9, 15, 16; stepwise values - - 
2, 3, A, 6, 8, 9, 10, 11, 12, 16; principal axis loadings — 1, 2, A, 5, 7, 10, 

12, 13, 15, 16, 17; variable-DF correlations -- 1, 2, 3, A, 5, 6, 8, 9, 15, 16, 
The corresponding proportions of correct classifications using these subsets 
were .637, .657, .642, and .623, while the proportion yielded by the subset 
detennined by the method involving MLFA was .602. Again it could not be con- 
cluded that these six iubsets differed in classification accuracy. 

Discussion 

When selection was based on high principal axis loadings the resulting 
loss in discrimination power was highly significant with both sets of data. 

That this scheme selected a subset of variables with lower discriminatory 
power may be expected. As Bargmann (1968, p. 574) states, . . it can be 
argued that variables which have the highest correlation against the first 
principal component contribute most strongly to discrimination among individual 
experimental units." In the analysis considered here, however, it is the 
be tween- groups variation which is of interest, net the between-unit s variation. 
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The popularity of the use of weights applicable to variables in standard 
form (herein called ’^beta weights”) is probably due to the familiarity of 
multiple regression techniques to many researchers. The problem of instability 
of beta weights over repeated sampling exists in discriminant as well as in 
regression analysis (Huberty and Blommers[in preparation (b) ] . Bock and Haggard 
(1968, p. 118) elaborate on the statement that, ”Even when standardized, the 
coefficients of the disciriminant function(s) do not always reflect closely the 
direction or magnitude of effects in co r respond inr variables." Further, the 
relative ordering of variable contribution is not preserved after some variables 
are deleted; it is necessary to recompute the weights with the fever number of 
variables . 

On the deletion of nonsignificant variables. Grizzle (1970, p. 319) comments 
tiiat . .the chance of finding significant differences by a multivariate test 
if lessened when dependent variables which do not contain information about 
differences among treatments are included in the analysis." It has been argued, 
however, that even though a variable may produce a nonsignificant univariate 
statistic, because of the discriminator intercorrelations this variable may 
appear relevant to the discrimination when included in the whole set of discrimi- 
nators. 

As in regression analysis there are some concerns over the use of the stepwise 
procedure in discriminant analysis. IVo such concerns are 1) tlie estimated ordering 
of the variables by this method may be biased (because of the presence of "etror 
suppressor" variables), and 11) this method will not necessarily lead lo selection 
of the "best" subset a gi ;en size. 
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If, in iTcducing the njnber of predictors, interest is centered on obtaining 
a subset that best represents the set of original variables, then the factor 
analy t ic-discrinina tory methoa ay be the most appealing. Usua31y, however, 
the primary concern is one of arriving at a subset which will have riearly the 
same discriminatory po;^er as the original set; representativeness and high 
discriminatory power will not necessarily simultaneously characterize the subset 
chosen. 

There are two characteristics of the variable-discriminant function correlations 
which ought to be mentioned. The first is tnat the rank-order of these correlations 
is unchanged after some variables have been deleted. Secondly, the rank-order 
remains invariant under a monotonic transformation of the variables. 

Proportions of correct classifications were obtained for subsets of various 
sizes determined by four selection criteria to further assess these criteria in 
terms of the relative discriminatory power of the chosen subsets. Subsets of 
size three to size p-1 {12 when k=3, and 16 when k=5) were considered. The 
proportions obtained for the four selection criteria are given in Tables II 
and 12. In the three group case all subsets, with a single exception, determined 

-Insert, Tables 11 and 12 About Here- 
by the stepwise procedure exhibited at least as much disc ri pi na t cry power as 
any of the subsets determined by any of the other three criteria. Of tlie fourteen 
subset sizes considered in Che five group case the stepwise procedure deternined 
the best subset eight tines. 

blicn investigating the relative discrim natory po'^^^r of selected sub.scts 
either between or v’i thl a selection procedures we concede that these procedures 
do not ordinarily yield the best subaet of retained ^’ariabl^,s of a gicen s’ze. 




22 . 



- 21 - 



Further, reLention or (deletion of some variables is often made after such 
considerations as availability, reliability, validity, and cost of measures 
as well as contribution to discrimination. 

^qncju^^_ 

Of course, a definitive answer to the question of vjliich of ^he selection 
methods studied is ’’best” cannot be given from the results of such an empirical 
investigation. Nonetheless, the present study does shed some light on tVie 
relative merits of some frequently used methods of determining variable con- 
tribution to discrimination. If a single selection method from those studied 
were to be chosen as best, based on the data used, it would be the stepwise 
method. There quite possibly Is a method that may be superior to any studied 
here, however. To date, a method of variable selection in regression analysis 
which appears to have considerable promise is that given by Hocking and Leslie 
(1967). in searching for the best subset of size q (<p) a considerable number 
of the {^) possible subsets is eliminated. The criterion used in finding the 
desired sulset from those remaining is the reduction in the regression sum of 
squares due to removing a set of p-q variables. Applying their techniques 
CO discriminant analysis might involve a quadratic f denoted by Pao (1952, 
p. 257) as and which may be termed a "generalized liahalanobis D , This 
approach deser “ further study* Another possibility which m.erlts considerat i on 
is that of determining criteria to be errployed in a "back' \srj " selection scheme 
(in which variables are successively discarded one at a time from tJie original 
full set)--this is in contrast to the BMD "forward" procedure. 
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^Profesoor barren G. FinJley is acknowlf-dged for reading an earlier 
draft of the manuscript. 

2 

Some researchers consider discriminatory analysis and (mul tivaria te) 
classificatoty analysis as separate or even independent techniques. 

3 

A modified version of FC/'dI, by Professor R. E. Bargmann, was used. 
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I leans and Standard Deviations 
Three-Croup Case^ 





Variable 




: leans 




No. 


Nape 


Gioup 1 
(K=35) 


Cl roup 2 
(N=8l) 


Group 3 
('.•=37) 


1 


High School English Cumulative 
Grade Point Average 


2.91 

(0.50) 


3.2^' 

(0.50) 


3. 33 
(0.49) 


2 


High Schcol Mathematics Cur ilative 
Grade Point Average 


2.52 

(0.70) 


2.36 

(0.80) 


2 . 69 
(0.80) 


3 


High School Social Science Cumulative 
Grade Point Average 


2.S9 

(0.4S) 


3.34 

(0.5.2) 


3.21 

(0.55) 


4 


High School Natural Scie'ice Cumulative 
Grade Point A>'erage 


2.6S 

(0.54) 


3.07 

(0.60) 


2.95 

(0.69) 


S 


Tlie Number of Semesters of High Scliool 
French Taken 


3.17 

(0.98) 


4.23 

(0.76) 


4.97 

(1.12) 


6 


High Scliool French Cumulative Grade 
Point Average 


2.68 

(0.69) 


3.20 

(0.64) 


3. 15 
(0.67) 


7 


American College Testing Program- - 
English (Standard Score) 


22.43 

(2.00) 


23.80 

(2.86) 


25.03 

(2.51) 


8 


Amicrican College Testing program 
Mathematics (Standard Score) 


23.66 

(5.73) 


24.70 

(4.90) 


24.27 

(5.35) 


D 


American College Testing Program 

Social Sciences (Standard Score) 


25. U 
(3.96) 


26.59 

(3,14) 


27.52 

(3.68) 


10 


American College Testing Program 

Natural Sciences (Standard Score) 


23.31 

(4.16) 


24.63 
(4 57) 


26.05 

(4.61) 


11 


Tlie ETS Cooperative French Placcncnt 
Test Scorc--Aural Con]irchcnsion 


14.34 
(4. 78) 


10.27 

(3.62) 


25.27 

(3.96) 


12 


The ETS Cooperative French Placement 
Test Scoro--Grammar 


40.86 

(7.55) 


52.88 

(7.22) 


. 63.76 
(4.01) 


13 


The Number of Semesters Since Last 
High School French Course I'as 


2.00 

(2.16) 


1 .40 
(1.68) 


0.49 

(0.99) 



Taken 



a . 

Standard deviations in parentheses 



No 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 



Table 2 



Megans and Standard Deviations 



Variable ^ 


ive- Group 


Case*" 


Means 




Nanie 


Group 1 
(;.-i77) 


Group 2 
(N=26) 


Croup 3 
(M=75) 


Group 4 
(M=52) 


Literature Information 


1 1 . 75 
(4.08) 


13.12 

(2.94) 


13.53 

(4.59) 


14.38 

(4.02) 


Social Studies Information 


13.71 

(5.16) 


14.50 

(4.64) 


15. 37 
(5.11) 


17.44 

(4.70) 


English Total 


79. S6 
(19.38) 


5? 69 
(19.10) 


81.39 

(14.97) 


85.56 

(15.49) 


?4echanical Reasoning 


9.58 

(4.13) 


8.31 
(4.. 55) 


12. 37 
(4.03) 


10.83 

(4.31) 


Visualization in 3 Dimieiijion 


8.05 

(3.32) 


7 ‘1 2 
(3.46) 


9.56 

(3.19) 


8.29 

(3.33) 


MathQnatics Infornation 


11.53 

(5.55) 


11.23 

(3.S5) 


1^.63 

(7,22) 


15.60 

(6.88) 


Clerical ^Perceptual Speed 


74. K 
(28.09) 


71 . 58 
(28. 71) 


76.71 

(25.67) 


71.92 

(23.36) 


Physical Science Interest 


13. 31 
(8.32) 


10.73 

(7.64) 


15.53 
(5. 64) 


15.98 

(7.92) 


Literary^- Linguistic Interest 


16.81 

(8.66) 


17.62 
(8.. 85) 


15.43 

(l('.b2) 


17.29 
(9. CO) 


Business Management Interest 


16.23 

(7.62) 


15.10 

(5.50) 


15.17 

(8.47) 


15.65 

(7.29) 


Computation Interest 


14.92 

(8.89) 


(7.97) 


1.3..36 

(8.37) 


14. IS 
(8.18) 


Skilled Trades Interest 


n.OG 

(6.53) 


7.50 

(3.9;!) 


11.63 

(7.66) 


0.27 
(6.4 2) 


Sociabi 1 ity 


6.33 

(.3.05) 


6.42 

(2.67) 


6.. 71 
(3.03) 


6.40 

(5.17) 


Jnpulsiveness 


1 .94 
(1 .41) 


1 . 85. 
(1-5V) 


1.87 

(1.69) 


1.70 

(1.44) 


^^^ture Personality 


10.88 

(4.78) 


10. 35. 
(3.61) 


11.35 

(5.33) 


12.35 
(4. S3) 


Curriculum 


3.33 

(1.42) 


3. 77 
(1.11) 


3.42 
(1 .61) 


3.98 

(1.50) 


SocioecoDoni c Composite 


92.82 

(13.13) 


96.81 

(21.18) 


94 96 
(17. 53) 


98.54 

(16.24) 



'^Standard deviations in parentheses 



Group 5 

‘ 15.94 
14.53) 

18. 56 
(4.45) 

85.36 

(13.23) 

12 . 3 :. 

(^1.43) 

V.79 

(3.40) 

20.39 

(7.75) 

78.31 

(24.93) 

19.67 

(10.07) 

21.10 

(9.30) 

18.58 

( 8 . 21 ) 

15.83 

(9.27) 

8 . 64 
(6. IS) 

6.93 

(7.91) 

2.97 

( 1 .^ 0 ) 

13. .^6 
(5.^7) 

4. W 
(1.S5) 

100.20 

(24.31) 
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'able 3 





Tarti al 


nata for t]*c 


Tl\rcc- Group Cdse 






Beta 


Univariate 


Variable vs 


DF Correlation 


Vari ibl€; 


Weight 


F-rat io 


Total Group 


Within Groups 


1 


0,43 


8.50** 


0.34 


0.18 


2 


-2.46 


2,40 


0.10 


0.05 


3 


2,02 


9.28** 


0.26 


0.14 


4 


6.18 


5.84** 


0.21 


0.11 


5 


3.42 


35.60** 


0,66 


0.41 


6 


0.30 


8.23** 


0.29 


0.16 


7 


2.18 


8.84** 


0,37 


0.20 


8 


0.84 


0.58 


0,05 


0.03 


9 


-2.62 


3. 72* 


0.25 


0. 13 


10 


-1.74 


3.36* 


0.24 


0.12 


11 


6.43 


67.87** 


0.78 


0.56 


12 


7.28 


105.86** 


0.88 


0.70 


13 


1.87 


7.97** 


-0,35 


-0.19 



* p C »05 

** p< ,01 
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Table 4 



Ordering of Variables 
Three-Group Case 



Beta 

Weights 


Univariate 

F-ratios 


Stepwise 

Values 


Variable vs DF 
Correlations 


12 


12 


12 


12 


11 


11 


11 


11 


4 


5 


A 


5 


5 


3 


5 


7 


9 


7 


3 


13 


2 


1 


9 


1 


7 


6 


7 


0 


3 


13 


6 


3 


15 


4 


13 


9 


10 


9 


2 


10 


8 


10 


10 


4 


1 


2 


8 


2 


6 


8 


1 


8 
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Table S 



i^otated Loading5 for tho 'Hi re e- Group Case 



.Maxi mura Likelihood Principal Axi s 



Variable 


I 


II 


III 


IV 


V 


I 


II 


III 


IV 


1 


80 


07 


03 


-13 


-Oi 


76 


11 


16 


26 


2 


-42 


12 


01 


-44 


- 64 


66 


40 


-19 


10 


3 


20 


11 


05 


-80 


25 


85 


02 


-02 


07 


4 


-17 


13 


-02 


-75 


-26 


80 


24 


-18 


14 


b 


00 


33 


06 


-15 


69 


-24 


11 


23 


-80 


6 


-01 


-02 


31 


-05 


-65 


66 


19 


21 


43 


7 


08 


08 


12 


66 


-90 


07 


52 


14 


64 


8 


-46 


31 


-10 


01 


-72 


21 


70 


-28 


18 


9 


03 


61 


09 


-09 


-02 


24 


73 


16 


-09 


10 


-03 


74 


-02 


07 


-14 


08 


85 


01 


05 


11 


-01 


34 


31 


45 


07 


-44 


34 


49 


-07 


12 


01 


25 


46 


13 


18 


-02 


12 


74 


-21 


13 


OS 


07 


-59 


-06 


-05 


-08 


30 


-75 


1 

oo 



^Decimal points oTTsitted 
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Table 6 



T!ie Five Variables Selected 
and Proportions of Correct Classifications 
Three-Group Case 



Beta 

Weights 


Univariate 

F~ratios 


Stepwise 

Values 


Principal 
Axis Loadings 


ML (viadings vs 
DF Correlations 


Variable vs 1) 
Correlations 


4 


3 


3 


3 


1 


5 


S 


5 


4 


5 


3 


7 


9 


7 


5 


10 


5 


11 


11 


11 


11 


12 


11 


12 


12 


12 


12 


13 


12 


13 


Proportion 












.876 


.899 


.895 


.810 


.882 


.869 
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Table 7 



Partial Data for the Five-Group Case 



Variable 


Beta 

Weight 


Univariate 

F-ratio 


Variable vs 
Total Group 


DF Correlation 
Within Groups 


1 


-6.62 


26.15** 


0.61 


0,53 


2 


36.60 


30.25** 


0,66 


0.57 


3 


-24.78 


10.20** 


0.41 


0.33 


4 


-2.09 


14.60** 


0.38 


0.32 


5 


-3.69 


9.90** 


0.35 


0.27 


6 


91.02 


51.52** 


0.82 


0. 76 


7 


0.09 


1.28 


0.11 




8 


36. 12 


16.38** 


0.49 




9 


57.13 


9.15** 


0.37 




10 


44.93 


4.84** 


0.26 




11 


-30.83 


1.53 


0.11 




12 


-65.30 


6.36** 


-0.27 


- 


13 


3, 27 


1.41 


0. 15 




14 


-4.31 


0.73 


0,07 




15 


9.15 


10.21** 


0.41 




16 


19.19 


13.41** 


0.46 




17 


-1.75 


3.55** 


0.24 





**p c;.oi 
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Table 8 



Ordering of Variables According 
to Pour Criteria 
Five-Group Case 



Beta 

Keights 


Uiu variate 
F-ratios 


Stepwise 

Values 


Variable vs OF 
Correlations 


6 


6 


6 


6 


12 


2 


2 


2 


10 


1 


4 


1 


y 


S 


9 


8 


2 


4 


12 


16 


8 


16 


10 


15 


11 


15 


3 


3 


3 


3 


8 


4 


16 


5 


11 


Q 


15 


9 


16 


B 


1 


12 


1 


12 


14 


10 


5 


10 


5 


r/ 


7 


17 


13 


11 


IS 


13 


4 


13 


14 


11 


17 


7 


17 


7 


7 


M 


13 


14 
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Table 9 



Rotated Loadings for tlie Five^Group Case"^ 



Maximum Likelihood 



Principal Axis 



riable 


I 


II 


III 


IV 


V 


VI 


VII 


VIII 


r 


II 


m 


rv 




VI 


1 


67 


-12 


11 


-11 


07 


-05 


05 


-01 


16 


-12 


co - 


ID 


09 


87 


2 


73 


-01 


11 


00 


01 


04 


04 


10 


33 


-05 


ll 


04 


-09 


77 


3 


06 


-06 


19 


-50 


06 


-02 


-08 


06 


43 


-12 


11 


06 


57 


' 37 


4 


-05 


05 


14 


-10 


00 


12 


00 


81 


85 


10 


-01 


-05 


01 


15 


5 


-13 


06 


06 


-29 


08 


11 


03 


58 


76 


05 


-02 


00 


20 


07 


6 


13 


09 


31 


-28 


06 


-00 


-30 


32 


67 


09 


08 


19 


16 


37 


7 


-28 


-02 


08 


-47 


12 


-04 


02 


01 


22 


-06 


06 


10 


SI 


-14 


8 


11 


47 


09 


15 


-06 


-14 


-li 


45 


41 


68 


-03 


00 


-15 


16 


9 


06 


17 


07 


-10 


04 


-43 


09 


-16 


-39 


53 


05 


08 


41 


36 


10 


02 


66 


01 


11 


-01 


-27 


03 


01 


-Vi 


83 


11 


)3 


-01 


-05 


11 


-10 


58 


16 


07 


-02 


-15 


-19 


02 


09 


72 


11 


13 


04 


-16 


12 


-08 


48 


-16 


07 


-12 


-23 


-10 


33 


09 


74 


-09 


-16 


-LM 


-10 


13 


-14 


07 


50 


05 


03 


03 


34 


-02 


-04 


01 


84 


i)0 


1 3 


-03 


14 


-01 


00 


06 


-13 


10 


-10 


40 


10 


09 


08 


35 


-11 


J9 


■> ? 


15 


08 


10 


59 


14 


02 


06 


03 


-01 


12 


04 


76 


11 


-07 


20 


16 


-00 


00 


-01 


-02 


86 


Os 


-01 


-04 


00 


05 


04 


87 


03 


11 


17 


-10 


- J 9 


00 


-08 


61 


-04 


07 


06 


04 


-02 


00 


84 


06 


02 



I^ecimal points onittcd 
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Table 10 



The Seven Variables Selected 
and F'roportions of Correct Classifications 
Five -Group Case 



Beta Univariate 

Weights F-ratios 



Stepwise Principal ML Loadi vs Variable vs FfF 
Val ues Axis Loadings OF Corr el ations Correla tions 



2 

6 

8 

9 

10 

11 

12 

Proportion : 
.618 



1 

2 

4 

6 

8 

15 

16 

.62 2 



2 

3 

4 
6 
9 

10 



.625 



1 
4 
7 
JO 
1 3 
16 
17 

.577 



1 

2 

6 

8 

9 

12 

10 

.60? 



1 

2 

3 

6 

8 

i 

16 

.612 
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Table 11 



Prcportioiis of Correct Classification 
lb rce- Group Case 



Number of 
Variables 


Bet a 
Wci|ihts 


Univariate 

F-ratios 


Stepwise 

Values 


Variable vs DF 
Correlations 


3 


.876 


.850 


.876 


.830 


4 


.882 


.882 


.882 


.882 


S 


.867 


.889 


.895 


.869 


6 


.869 


.889 


.908 


.889 


7 


.895 


.889 


.902 


. 902 


8 


.90S 


.889 


.908 


.889 


9 


.908 


.915 


. 908 


.902 


10 


.90S 


.922 


.935 


.915 


11 


.928 


.928 


.928 


.928 


12 


.941 


.922 


. 948 


.922 


15 


.941 


.941 


.941 


.941 
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Tabic 12 



Proportions of Correct Classifications 
Five -Group Case 



Numbei' of 
Variables 


beta 

^/eights 


Univariate 

F-ratios 


Stepwise 

Values 


Variable vs 
Correlation 


3 


.583 


.575 


.575 


.575 


4 


.588 


.600 


.585 


.600 


5 


.595 


.599 


.592 


.593 


6 


.592 


.607 


.603 


.610 


7 


.618 


.622 


.625 


.612 


8 


.615 


.608 


. 622 


.608 


9 


.627 


.617 


.653 


.623 


10 


.637 


.623 


.657 


.623 


11 


.650 


.647 


.670 


.647 


12 


.685 


.669 


.672 


.668 


13 


.697 


.6,82 


.677 


.632 


14 


. 705 


. 705 


.713 


.698 


15 


.713 


.710 


.74 7 


.710 


16 


. 728 


. 733 


.742 


.733 


17 


.750 


. 750 


.750 


.750 



o 

ERIC 



